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RECOVERING FOURIER COEFFICIENTS 
OF MODULAR FORMS 
AND FACTORING OF INTEGERS 

Sergei N. PreobrazhenskiQ 

It is shown that if a function defined on the segment [—1,1] has sufficiently good approxi- 
mation by partial sums of the Legendre polynomial expansion, then, given the function's 
Fourier coefficients c n for some subset of n G [711,712], one can approximately recover them 
for all n E [711,712]- As an application, a new approach to factoring of integers is given. 
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1. Introduction. Let a function f(x) being a polynomial of degree K in the interval 
[— |, I) be continued periodically to the entire real axis with period 1: 



f(x) = b + bix + . . . + b K x K , x G 



1 1 

'2' 2 



b K ^0, f{x + l) = f{x) 



A question is this: Can we find all Fourier coefficients of the function f(x) if we know K + 1 
values of the coefficients a 



-pen 



., c PK l Obviously, one may set up the system of linear equations 



/ °i,i> 

a 2,l, 



Cf2,A"+l 



fb \ 

bi 



(c \ 

c pi 



a>K+i,K+i/ \b K J 



Y*-p;, 



So if we prove that the matrix of the system is nonsingular then we can uniquely identify the 
polynomial and find the remaining Fourier coefficients of the function f(x). 

In this note we show that if a function defined on the segment [—1, 1] has sufficiently good 
approximation by partial sums of the Legendre polynomial expansion, then, given the function's 
Fourier coefficients c n for some subset of n e [ni,^], one can approximately recover them for 
all n G [711,712]. As an application, a new approach to factoring of integers is given. Among 
factoring algorithms now in use the fastest ones are: the elliptic curve method, the quadratic 
sieve and the number field sieve method. The quadratic sieve algorithm was introduced by 
Pomerance [1, 2] in 1981. The heuristic complexity of the algorithm is 

exp ((1 + o(l))(logn) 1/2 (loglogn) 1/2 ) 

arithmetic operations, where n is the number to be factored. The elliptic curve method of 
H. Lenstra [3, 4] appeared in 1986. If p denotes the least prime factor of n, then the expected 
number of operations required to factor n with the elliptic curve method is 



exp ((v / 2 + o(l))(logp) 1 / 2 (loglogp) 1/2 ) (log n] 



o. 



(1) 



where C2 is a positive constant. The number field sieve was suggested in [5]. The heuristic 
complexity for factoring n via this method is 



exp((C 1 + o(l))(log72) 1/3 (loglogri) 2/3 ) . 
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The main idea underlying the approach suggested in this note is to compute Fourier coefficients 
of modular forms that arise from elliptic curves (see [6, 7]). 

Here is an outline of the method. It is known that certain modular forms have integer 
Fourier coefficients c n , and these coefficients satisfy specific equations (see below). Moreover, 
the famous Shimura-Taniyama conjecture (proved by Wiles for the case of semistable elliptic 
curves) states that under certain conditions the numbers a p — defined in terms of the orders over 
Z p of a fixed elliptic curve E over Q — coincide with the Fourier coefficients c p of a modular form 
of the aforementioned type. The numbers a p are defined by the equation: #E(Z p ) = p + 1 — a p . 
One may use these properties to factor n = pq. Let p and q be odd primes of about the same 
size (such n's are called RSA- numbers). Choose arbitrary elliptic curve E over Q. If for the 
corresponding modular form we could find the coefficients c n and c n 2, factor them, and thereby 
find, say, c p and c p 2, then from the aforementioned equations for the coefficients we would find 
p = {c p ) 2 — c p 2. The problem is to find c n and c n 2 when the factorization n = pq is unknown. 
(If the factorization was known, we would find c n and c n 2 by computing c p = a p and c q = a q 
via the Schoof algorithm for the curve E.) Thus, we wish to reduce the problem of factoring n 
to the problem of factoring c n and c n 2 that depend on a randomly chosen curve E. 

Assuming that the modular form on the interval [—1, 1] has sufficiently good approxima- 
tion by partial sums of the Legendre polynomial expansion, one could try to find c n and 
c n 2, factorization of n being unknown, via the "approximation" method as it is enunciated 
at the beginning of the section. One represents Fourier coefficients of the modular form as 
integrals, uses the polynomial approximation and the Schoof algorithm for computing small sets 
of coefficients c p > and c p » for small sets of primes {p'} and {p"}, near n and n 2 , respectively. 
The coefficients in the "knots", the sets {p'} and {p"}, "interpolate" the coefficients in the 
"points" n and n 2 . 

2. Main theorem. Suppose r = p + icr, o = - is fixed, p G [—1, 1]. Write the function 
(p(r) = /(r)e _27rmr as the sum of its real and imaginary part: 

V (r) = f(r)e- 2 ^ = u a (p) + zv a (p). 

Suppose 

n < p[ = n + A x < p' 2 = n + A 2 < . . . < p' K = n + A K . 

For an integer A, denote by ca(/) the A-th Fourier coefficient of <f{r). It coincides with 
{n + A)-th coefficient of /(r): 

i 

2c A (/) = e 2 ^' n J (u a (p) + iv a {p)) e-^o dp. 
-l 

Take the finite Legendre polynomial expansion for the real and the imaginary part of v?(t): 

u a (p) ~ aiPi(p) + . . . + a K P K {p) = U K (p), 

v a (p) ~ ftPxO) + . . . + !3 K P K (p) = V K (p). [ ] 

Denote by C&(ct, ($) the approximation to the Fourier coefficient ca(/) obtained via the expan- 
sion fl2J: 

i 

2C A (a,P) = e 2 ^' n J (U K (p) + iV K (p)) e-*P*X>' dp. 
-l 
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Making successive substitutions A = Ai, A = A 2 , . . ., A = A K and integrations, we have the 
matrix equation (here e 27rA / n denotes the diagonal matrix with the entries e 27rAl//n , . . ., e 27rA if/ n ) 



/2C Al (a,0)\ 



^2wA/n 



\2C AK (a,P)J 



ai,i(Ai), a hK (A 1 ) 
.a K ,i(A K ), a K ,K{A K ), 



or 



- e 27rA/n Ais v 

— e -"-KxK 



\2C AK (a,(3)J 



\a K + i(3 K ) 



The entries of the matrix Akxk are Gegenbauer's integrals, a generalization of Poisson's inte- 
gral. These integrals can be cast in terms of Bessel functions of half-integer order, and we have 
the general formula: 



1 



-Jm+i/2(27rA k ). 



a k>m (A k ) = (-i) 



Theorem 1. The determinant det A^xk is well-defined when A k ^ and nonzero if Ai ^ 



7^ •?)■ ^/ we define Hankel symbols by 



(1/, 0) = 1, (i/,Z) 



(4z/ 2 - l 2 ) (4z/ 2 - 3 2 ) . . . (4z/ 2 - (2/ - l) 2 ) 



2 2l l\ 



for Z = 1,2,3,.. 



(3) 



t/ien we can wnto £/ie formula: 
det A 



- ^ I 11 (4^=2 I X 

^C 2J (2„+l/2,2 P -l) \ («1 

11 I" ) (4^-1 - "J (Jl A 



n G~i 



(4) 



J.^i<j^K 



Define Dk,a (E) when A 7^ to &e t/ie supremum of the set {Dk,a (E)} of numbers satisfying 
the condition: if max(\e[ + ie'/l, . . . , + < D K Ao (E) then we have 



4-I -2nA/n 
KxK 



and 

It follows that if 



( 2C Al + e[ + ie'[ \ 
\2C AK +e' K + ie'l ( J 

2C Ao (a + 5',P + 5")-2c Ao (f)\ < E. 



( ai + + <JJ + i5'l \ 
\a K + i/3 K + S' K + ib" K J 



- (Dk,Ao(E)Y 



then 



k=K+l 



a-1 -2wA/n 
KxK^ 



/2c Al (/)\ 

V 2c Ak(/)/ 



2max fc e 47rA fc/ 

/ cti + ifr + S[ + iS'{ 
\a K + ip K + 6' K + ib" K 



(5) 



(6) 



and 



\2C Ao (a + 5,P + 5 )-2c Ao (f)\<E. (7) 

So if we know the Fourier coefficients 2c Al (f), . . ., 2c& K (f), then we may recover the Fourier 
coefficient 2ca (/) (A 7^ 0) to the precision E. 

Remark. In this theorem, instead of using Legendre polynomials, one could use the simple 
powers x, x 2 , . . ., x K . But in practice, we cannot know that (J5J) holds, and with Legendre 
polynomials we may infer this from stabilization of the coefficients 

/ «i + ip! + S[ + i8'l \ 

\a K + ipK + S' K + iS'ic J 

in ([6]) as K grows. For simple powers such stabilization of coefficients may not occur. 

Proof. We now prove (jlj). Using the notation ([3]) and known Hankel expansions for Bessel 
functions of half-integer order, we deduce that afc jm (Afc) are polynomials in of degree m with 
the leading coefficients 



'k,m 



t!l! cos -»-I|U] 

(-0* 



(ra+l/2,ro-l) 



if m — 2/i — 1; 



(_l )sin (_(z?±l/g)£ _ |) ( _ 1) M-l (^+l/2,m r l) ) ifm = 2/i 



By elementary operations, the matrix Akxk may be transformed to the Vandermonde matrix. 
This completes the proof of (jlj). 

We now prove that the condition ([S]) implies ([H]) and (j7j). Define 



<p CT (p) = u CT (p) + iv a (p), $k(p) = U K (p) + iV K (p). 



We have 



I 1 

J \<PM - ®i<(p)\ 2 dp = J (<p a {p) - $ K {p)) (W(p)-Mrf) dp = 

-1 -1 

II 1 1 

Va {p)\ 2 dp- / <5> K (p)^(p)dp- / $^(p)<p a (p)dp + / |$^(^)| 2 rfp 



-1 



-1 



/ \^(p)\ 2 dp - E K + /^ 2 ) = E ojfcTl W + = ^(/) 



Let us estimate \2C Ak — 2c Ak (f)\. Using the Cauchy inequality and the condition (j5j), for 
1 < k < K we obtain the estimates 



1 

|2C Afc - 2c Afc (/)| ^ e 2 ^/" | |^(p) - ® K {p)\ dp < 



1/2 



J |^(p)-$tf(p)| 2 dp) <Z^, Ao (£), 



\-l 
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from which by the definition of Dk,a {E) we infer ([6]) and (J7|). This completes the proof of the 
theorem. 

3. Modular forms and modular elliptic curves. We follow the book [8]. 
An unrestricted modular form of level N and weight 2 is an analytic function / on H such 



that for all ( ° ^ j G r (iV) and r G H we have 



Since ^ J j J G To(A^) then /(r + 1) = f(r) for each modular form and every r. Thus / 



has a Fourier expansion, which is of the form 

f( T ) = ^2c n e 

Here 



2-nin.T 



n=0 



f(r)e- 2mnT dp, 

where r = p + ia, and a > is fixed. For cusp forms, cq = 0. 

The following theorem conveys information about the magnitude of the modulus of a cusp 
form /(t) and the Fourier coefficients c n . 

Theorem (Deuring). Let f G ^(iV) /mue q-expansion (jSJ) at t/ie cws^ oo. TTien i/ie function 
<p(r) = \ f(r)\o~ is bounded on H and invariant under Tq(N) . Furthermore, we have \c n \ ^ Cn. 

Suppose / G S2{N) is an eigenform, normalized so that the q expansion (jHJ) has c\ — 1. 
Then for r ^ 1 the Fourier coefficients c„ of / satisfy 

CprCp = Cpr+i + pCpr-i, for j> prime, p \ N; (9) 
c p r = (cp) r , for p prime, p \ N; 

CmC n = c mn , gcd(m, u) = 1. 

The following statement is the Shimura-Taniyama conjecture, proved by Wiles for the case 
of semistable elliptic curves (i.e. for squarefree N). 

Proposition. If E is any elliptic curve defined over Q, if N is its conductor, then there 
is a normalized new cusp eigenform f of level N and weight 2, whose Fourier coefficients c n 
are integers and such that for every prime p not dividing N c p = a p (where a p is defined by 
#E(Fp) = p + 1 — a p , #E(F p ) being the order of the group of the elliptic curve E over F p ). 

Now we give an example of a modular form associated with an elliptic curve. 

Example (Hecke). Define 

A(r) = (27r) 12 g JJ (1 - <f) 24 and V (r) = -£=A(t)£. 



v '2tt 

n=l v 

Then it follows that /(r) = ^(llr) 2 ?7(r) 2 is a new cusp eigenform of weight 2 and level 11. It 
is associated with the elliptic curve E 

y 2 + y = x 3 - X 2 - WX - 20, 
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which for p ^ 2, 3 can be given the form 

, 31 2431 



4. Elliptic curves over finite fields. The following theorem gives a range for the order 
of an elliptic curve group defined over a finite field. 

Theorem (Hasse). Let E be an elliptic curve over Q with integer coefficients and the 
discriminant A. For each prime p \ A, let E(W P ) be the reduction of E modulo p. Then 
\p+l-#E(W p )\<2^p. 

We need two celebrated algorithms related to elliptic curves over finite fields: the Schoof 
algorithm [9] for elliptic curve point-counting and Lenstra's elliptic curve method for factoriza- 
tion, mentioned in Section 1. For prime q > 3 and elliptic curve E(¥ q ), the Schoof algorithm 
computes j^E{¥ q ) in complexity O (log 8 g) . The Lenstra method factors a composite n. If p 
denotes the least prime factor of n, then this method has the complexity ([1]). 

5. Smooth numbers. A positive integer is said to be y-smooth if it does not have any 
prime factor exceeding y. Let ip(x,y) denote the number of ^/-smooth integers among positive 
integers n ^ x. 

Theorem (Canfield-Erdos-Pomerance [10]). The estimate il)(x,x l l u ) = xu~ u+ °^ holds 
uniformly as u — » oo, u < (1 — e\) log x/ log log x ; E\ > 0. 

6. The central result and the algorithm. 

Theorem 2. Assuming that the method discussed in Section 2 for computing the Fourier 
coefficients of a modular form associated with an elliptic curve is correct and runs in polynomial 
time, the heuristic complexity estimate for factoring an integer n with the least prime factor p 
(p 2 \ n) is 

exp (c 1 (A;)(logj9) 1/fc (loglogp) 1 - 1/fc ) (logn) C2(fc) 

arithmetic operations for any fixed positive integer k; here Ci(k), 02(h) are positive constants 
which may depend on the choice of k. 

We now describe a recursive algorithm having the claimed complexity estimate. First, we 
show that the following method gives the complexity estimate 

exp (ci(3)(logj9) 1/3 (loglogp) 2/3 ) (logn) C2(3) 

(this method corresponds to the case k = 3). 

Step 0. Choose a smoothness parameter B x p l ^(^° ep ^ /3 (iogiogp) / 3 )^ w i ieTe p j s fae least 
prime factor of n to be factored. (In fact, since we do not know p to begin with, we are 
instructed to start with a low Bq value and then to run the algorithm with B = Bq, B = 2Bq, 
B = 4£> , • • ., choosing for each B about 

exp ((v / 2 + o(l))(log J B) 1/2 (loglog J B) 1/2 ) 

random elliptic curves in Step 1.) 

Step 1. Choose random elliptic curve E over Q with integer coefficients. 

Step 2. Compute the Fourier coefficients c n and c n 2 of the associated modular form with 
the method discussed in Section 2. 

Step 3. Try to factor these coefficients with the elliptic curve method, hoping for a successful 
event: the largest prime factor of the coefficient c p \ c n and the largest prime factor of the 
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coefficient c p 2 | c n 2 both do not exceed B. For each c\ \ c n and C2 | c n 2 compute d = gcd((ci) 2 — 
C2, n) and, if 1 < d < n, return the nontrivial divisor d of n. 

Step 4. Failure: goto Step 1 (or give up on the factorization attempt). 

Assume that the Fourier coefficients c p and c p 2 of the modular form associated with the 
chosen elliptic curve (where c p = O (y/p) and c p 2 = 0(p) by the Hasse theorem and the 
relation (|9])) are 5-smooth with the same probability as random integers chosen from the 

3 

respective intervals. Then we get i?-smooth coefficients c p and c p 2 expecting about u^ u , where 
u = (logp) 1 / 3 (loglogp) _1//3 , elliptic curves, with the attempt to find these coefficients in Step 3 
requiring 



exp ((v / 2 + o(l))(log J B) 1/2 (loglog J B) 1/2 ) (log 



; n 



2 



arithmetic operations. Multiplying the expressions, we obtain the estimate for the complexity 
of the algorithm in the form 



exp ( Cl (3) (log p) 1/3 (log log p) 2/3 ) (log 



,ca(3) 



If we now use this algorithm instead of the elliptic curve method in Step 3 and choose the 
optimal smoothness parameter, then we get an algorithm having complexity 



exp (ci(4)(logp) 1/4 (loglogp) 3/4 ) (log 



n 



,ca(4) 



Furthermore, if we use k + 1 levels of recursion, choose B x ^/(Qosp) /( + 3 (kg kg p) /( + ') 
and apply the algorithm with k levels of recursion, having complexity 



exp (ci(fc)(logp) 1/fc (loglogp) 1 - 1/fc ) (lo K n) C2 ^ 



then we arrive at the following complexity estimate: 

((log P ) 1 /^ 1 Hloglog P )- 1 /(^^^ logp)1/(fc+1)(loslogp) - 1/(fc+1) X 
x exp (c(ifc) (bgB) 1/fc (log log S) 1_1/fc ) {\ogn) c '^ k) = 
= exp (c' 1 (A;)(logp) 1/(fc+1) (loglogp) 1 - 1/(fc+1) ) x 
x exp (c' 1 '(A;)(logp) (1/fc)(1 - 1/(fc+1)) (loglogp) 1/(fe(fc+1))+1 - 1//£ ) ilognf'^ 
= exp (ci((fc + l))(logp) 1/(fe+1) (loglogp) 1 - 1/(fe+1) ) (logn) C2((/c+1)) . 
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